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Abstract: What are the cognitive and motivational consequences of adapting an agent metaphor 
in multimedia learning? The present paper reviews a set of studies designed to test the hypothesis 
that the presence of animated pedagogical agents in multimedia environments can promote deep 
learning. This was done by first, comparing the learning and motivational outcomes of students 
who learned in the context of social -agency was to those of students who learned in a more 
traditional text and graphics context. Second, the particular features of the social agency 
environment were manipulated to examine which of its attributes are most important in the 
promotion of meaningful learning. The theoretical and practical implications of the findings are 
discussed. 



Interface Agents and the Learner Experience 

Animated software agents figure predominantly in instructional design (Lester et al., 1998). The 
most common agent interface consists of an animated face, a cartoon character, or a human-like virtual 
agent who has the task to assist the user, to engage the user into a conversation, to educate the user, or to 
instruct the user to perform a certain task (Bradshaw, 1997). The argument for using highly visible agents 
generally relies on the assumption of anthropomorphism-the fact that people unconsciously ascribe mental 
states to computers and are quite adept at relating to and communicating with other people (Laurel, 1 997). 
However, despite the apparent potential of the agent metaphor, it is necessary to investigate the cognitive 
and motivational effects of agents’ presence in human-computer interaction. Agent design is sometimes 
centered upon technological capacity rather than research-based principles (Bradshaw, 1997; Genesereth & 
Ketchpel, 1994; Laurel, 1990; Maes, 1991). As Erickson states, “So far it looks like the agent metaphor is 
more trouble than its worth.. .Far more research is needed on how people experience agents. ..very simple 
cues like voice may be sufficient to invoke the agent metaphor.” (Erickson, 1997, p. 91). 



A Cognitive Theory of Multimedia Learning 

By beginning with a cognitive theory of how learners process multimedia information, it is possible to 
make predictions about how the different attributes of an agent metaphor may affect learning. The proposed 
cognitive theory of multimedia learning is based on the following assumptions: (a) working memory includes 
independent auditory and visual working memories (Baddeley, 1986); (b) each working memory store has a limited 
capacity (Sweller & Chandler, 1994); (c) meaningful learning occurs when a learner selects, organizes, and connects 
corresponding verbal and non-verbal information (Paivio, 1986); and humans actively engage in cognitive 
processing in order to construct a coherent mental representation of their experiences (Moreno & Mayer, 2000a). In 
the remaining sections 1 will offer a set of predictions derived from cognitive theory for each one of the reported 
studies. 



An Environmental Science Scenario 
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The multimedia learning environment used in this study is based on the computer program "Design-A- 
Plant", developed by the Multimedia Laboratory at the Department of Computer Science of North Carolina State 
University (Lester et al., 1998; Moreno, Mayer, & Lester, 2000). In this program, the student travels to an alien 
planet that has certain environmental conditions (e.g., low nutrients, heavy rain) and must design a plant that would 
flourish there (e.g., designing the root, stem, and leaves). It includes an animated pedagogic agent who offers 
individualized advice concerning the relation between plant features and environmental features and feedback on the 
choices that students make in the process of designing plants. The program starts with the agent introducing the 
student to the first set of environmental conditions. Then, the agent asks the student to choose the appropriate root 
from the library of roots' names and graphics shown on the computer screen. After the student had chosen a root, 
the same procedure applies to the stem and leaves, with the agent first asking the student to make a choice, and 
giving the student feedback afterwards. Once the student receives the last explanation on the leaves for each 
environment, he is taken to the next environment. The same procedure follows for the rest of the environments. 



Does Learning in an Agent -Based Environment Promote Deeper Learning than Learning 
Identical Materials in a More Traditional Text-and-Graphics Environment? 

To help answer this question, a preliminary study was conducted where the learning and motivational 
outcomes of students who learned about environmental science in the Design-A-Plant microworld (social-agency or 
SA group) was compared to the learning and motivational outcomes of students who received identical verbal and 
visual materials in a text -and graphics environment (no social-agency or no-SA group). 



Method and Results 

The participants were 44 college students. The computerized materials consisted of two multimedia 
computer programs on how to design a plant. The S A version was the above described "Design-A-Plant" program, 
where students see a library of plant parts, pick the plant part that they consider appropriate for the respective 
environment, and receive spoken feedback in a conversational style from the agent (Lester et al., 1998). The no-SA 
version was presented with the same library plant parts and explanations than the SA version, but the agent’s image 
was deleted throughout the program. In addition, students in the no-SA version were not allowed to design the plant 
before reading the verbal explanations, but rather received the explanation directly in a monologue style, similar to 
when science material is read from a text book. 

After interacting with the respective program, participants were tested on three measures: retention--in 
which students were asked to name as many types of roots, stems, and leaves as they could remember; problem- 
solving transfer--in which students were asked to solve new problems based on the principles learned; and program 
ratings--in which students were asked to rate their level of motivation, interest, understanding, and the perceived 
difficulty and friendliness of the lesson. 

Do students who learn interactively with a pedagogical agent show deeper understanding from a multimedia 
science lesson than students who learn in a conventional environment ? The mean number of correctly recalled 
items by the SA group was not significantly different than the mean number of correctly recalled items by the no-SA 
group. The results suggest that when retention of factual information is the goal of the program, then environments 
that allow for interacting with an animated pedagogical agent are not warranted. Although these results 
demonstrated that both groups learned the basic factual information, students in the SA group did produce 
significantly more correct solutions on transfer problems than students in the no-SA group (p < .005). 

The findings are consistent with the idea that students who learn with agents work harder to make sense of the 
material than do students who learn in a more conventional text-and-graphics environment. This idea was supported 
by the comparison of students’ program-ratings: The SA group rated their motivation to continue learning and their 
interest in the material significantly higher than the no-PA group (p < .01). In sum, these findings give preliminary 
evidence in favor of using pedagogical agents as software mentors, and demonstrate a personal-agent effect in 
multimedia learning environments: Students are more motivated, interested, and achieve better transfer when the 
lesson is imparted by a pedagogic agent rather than by on-screen non-personalized text and graphics. 
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How Do Agent -Based Multimedia Environments Affect Learning? 



Reeves and Nass (1996) have provided convincing evidence that students view their contact with 
computer-based characters as social interactions. Congruent with this approach, students’ learning with the 
pedagogical agent could have been promoted by at least four social-cues: the agent’s image, the agent’s voice, their 
interaction, and the personalized language style. The next logical step was to investigate the role that each one of 
these cues played in the motivational and learning outcomes of the preliminary study. First, to determine the role of 
the auditory and visual presence of the agent, we varied whether the agent's words were presented as speech or on- 
screen text and whether or not the agent's image appeared on the screen, both with an animated fictional agent and a 
video of a human face. Second, to determine the role of the interaction between agent and student, we varied 
whether or not the student was able to participate in the lesson by designing the plant before receiving the agent’s 
explanations. Third, to determine the role of the language style of the agent, we varied whether or not the agent's 
explanations were provided in a personalized style (i.e. as dialogue or monologue) both using speech and on-screen 
text. 



The Role of Agents’ Visual and Auditory Presence 

The main argument in favor of including highly visual agents in the interface is based on interest theory of 
learning (Dewey, 1913). According to interest theory, students communicate better, become more interested, and 
therefore learn better and rate more favorably computer programs that include social cues--such as facial expressions 
or human voices, than those that do not include such cues (Rutter, 1984). Conversely, according to a cognitive theory 
of multimedia learning (Sweller & Chandler, 1994; Moreno & Mayer, 2000b), the inclusion of irrelevant adjuncts in a 
multimedia presentation-such as the image of an animated pedagogical agent, may divert the limited cognitive 
resources available for the processing of the relevant materials. As a result, learning and problem solving will be 
impaired. 

Respect to agent’s auditory presence, prior findings on modality effects in multimedia learning found that 
students learn better from visual and verbal presentations when the verbal information is presented as speech rather 
than as on-screen text (Mayer & Moreno, 1998, Moreno & Mayer, 1999, Mousavi, Low, & Sweller 1995). The 
advantages of speech over text have been interpreted as due to more effective working memory and relatively 
effortless maintenance of the auditory input in comparison to the visual input provided by text (Moreno & Mayer, 
1999). Thus, students should perform better on tests of retention and problem solving when they learn with the 
agent’s voice rather than on-screen text. The following two experiments were conducted to examine these 
hypotheses. 



Method and Results 

In the first study, 64 college students learned in one of four conditions: with or without the image of a 
fictional agent who gave narrated explanations to them and with or without the image of a fictional agent who gave 
explanations as on-screen text. In the second study, 79 students participated in the same four treatment conditions 
with the exception that the image and voice of the fictional agent were replaced by the video and voice of a human 
agent. The procedure was the same as for the preliminary study. 

Do students who are presented with the image of a pedagogical agent show deeper understanding from a 
multimedia science lesson than students who are not presented with the image? Experiments 1 and 2 did not 
provide evidence in favor of either interest theory or cognitive load theory. The results failed to confirm an image 
effect in program ratings, recall, and transfer: Students who are presented with the image of an agent do not rate the 
lesson more favorable, recall more, or are better able to use what they have learned to solve problems than students 
who are not presented with the visual presence of the agent. The image of the agent did not help or hurt students’ 
learning. 

Do students who communicate with a pedagogical agent via speech show deeper understanding from a 
multimedia science lesson than students who communicate with a pedagogical agent via on-screen text? The 
findings from Experiments 1 and 2 gave evidence in favor of students' communicating with a pedagogical agent by 
means of speech by demonstrating a modality effect in program ratings, recall, and transfer: Students who learn 
with the voice of an agent rate the lesson more favorably, recall more, and are better able to use what they have 
learned to solve problems than students who learn the same verbal information as on-screen text. In both 
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experiments, the mean program ratings for the narration groups was significantly higher than the mean program 
ratings for the text groups (g. < 0.05 for both experiments); the narration groups recalled significantly more than the 
text groups (g. < .005 and£< 0.005 for Experiments 1 and 2, respectively); and the narration groups gave 
significantly more correct answers in the transfer tests than the text groups (j> < .0005 and j> = .0001 for Experiments 
1 and 2, respectively). These results extend prior findings of modality effects in learning from visual and verbal 
materials to interactive, agent-based multimedia environments (Moreno & Mayer, 1999; Moreno et al., 2001). 



The Role of Students’ Interaction 

Experiment 3 tested the hypothesis that the main attribute promoting meaningful learning in an agent-based 
environment is students' participation. The goal was to determine whether the effects obtained the preliminary study 
could be attributed mainly to the difference in the level of interactivity between treatment groups. According to a 
cognitive theory of multimedia learning, a central part of the learning process occurs when students attempt to apply 
the instructional material to solve problems for themselves (Anderson, 1983; Moreno et al., 2001). Experiment 3 
compared an agent-based computer lesson where students were able design a plant for each environment before 
listening to the agent's explanations, with an identical lesson where students were not able to design plants during 
the interaction but rather listened to the agent's explanation directly. 



Method and Results 

The participants were 38 college students who learned with or without participating in the process of plant 
design. After being introduced to each environment, students in the participatory version (P) clicked on a plant part 
to design a plant before listening to the agent's explanation. Students in the non-participatory version (No-P) were 
presented with the same plant library but clicked on a "continue" button before listening to the same explanation. 
The procedure was identical to that of the prior studies. 

Do students who participate in the process of plant design show deeper understanding than students who 
learn with no participation? The results supported a cognitive theory of multimedia learning by demonstrating an 
interactivity effect in recall and problem-solving transfer: Students who learn by participating in the learning task 
with the pedagogical recall more and are better able to use what they have learned to solve far transfer problems 
than students who learn in a non-participatory agent-based environment. The P group recalled significantly more (j) 
= .01) and gave significantly more correct answers in the far transfer tests (g. = .04) than the No-P group. However, 
groups did not differ on program-rating scores. Overall, the findings are consistent with a cognitive of multimedia 
learning and allow us to conclude that participatory environments encourage the deep processing of the materials of 
a lesson by engaging students in an active search for meaning (Moreno & Mayer, 2000a) 



The Role of Agents' Language Style 

Past research has shown robust evidence for a phenomenon called the self referential effect , in which 
retention is facilitated by having people process information by relating it to aspects of themselves (Rogers, Kuiper, 
& Kirker, 1977). In Experiments 4 and 5, self-referencing was created by a personalized style of communication, 
where students were addressed directly and encouraged to believe that they were active participants in the lesson. 
According to a cognitive theory of multimedia learning, self-referencing may promote deep learning in two ways: 
first, by engaging students in the active elaboration of the materials and second, by using less cognitive effort to 
process verbal information when it is presented h a familiar style (i. e., normal conversation) rather than an 
unfamiliar style (i. e., monologue) of communication. 



Method and Results 

In Experiment 4, 39 college students learned either with a personalized conversation spoken by the agent or 
a non-personalized monologue spoken by the agent. In Experiment 5, 42 college students learned either with a 
personalized conversation displayed as text or a non-personalized monologue displayed as text. The procedure was 
identical to that of the prior studies. 
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Do students who communicate with a pedagogical agent via a personalized dialogue show deeper 
understanding than students who communicate via a non -personalized monologue? The findings from Experiments 
4 and 5 gave evidence in favor of students’ communicating with a pedagogical agent by means of a personalized 
conversation by demonstrating what w have called a self-reference effect in recall and transfer: Students who learn 
by communicating with a pedagogic agent via a personalized dialogue recall more and are better able to use what 
they have learned to solve problems than students who communicate via a non-personalized monologue. The mean 
number of ideas recalled for dialogue groups was significantly larger than for monologue groups (p < .005 and £ < 
.05 for Experiments 4 and 5, respectively) and the mean number of correct answers in the transfer test was 
significantly larger for the dialogue groups than for the monologue groups (p < .0001 for both experiments). In 
addition, Experiment 4 demonstrated a self-reference effect for program ratings. Students who learn via on-screen 
text rate the program more favorably when it is presented in a dialogue rather than a monologue style (£ = 0.05). 



Conclusion 

The reported results supported a cognitive theory of multimedia learning when three social cues are present 
in the interface: the agent’s voice, a personalized language style, and students’ interaction. Based on the assumption 
of limited cognitive resources, it was predicted that the introduction of the agent’s image in the computer interface 
would be detrimental to students’ learning. This prediction was not confirmed. According to cognitive load theory, 
a detrimental effect in learning occurs in the cases that students need to split their attention between mutually 
referring materials (Sweller & Chandler, 1994). However, in our studies, the agent's animated image was never 
presented simultaneously with other visual materials of the lesson. It is more likely that multimedia presentations 
containing simultaneous animations of the agent and graphics or text, would result to be detrimental rather than 
neutral to learning (Mayer & Moreno 1998, Moreno & Mayer 1999). 

On the other hand, several interpretations can be offered to explain the lack of an image effect. First, the 
voices used for the fictional and human agents in the first two experiments were extremely clear and expressive. 
When voices carry these qualities, it is less likely that facial expressions or lip movements will be helpful in 
understanding the instructional message. Second, if the goal of the instructional material is to teach procedural 
knowledge, such as how to make a machine work, the use of an agent’s image and gestures might play a crucial role 
by supplementing a conversation with pointing actions and gaze (Hanne & Bullinger, 1992). Third, the lack of an 
effect may reside in the scientific content of the lesson. It might be that for other subjects such as social sciences, 
learning with the image of an agent plays a fundamental role. Faces for example, can add vital information about 
the intensity and valence of the social events described (Ellsworth, 1975). More research is needed to investigate the 
role of agents’ visual presence in other multimedia learning situations. 

On the practical side, the present study has direct implications for instructional design. The reviewed 
studies offer encouraging evidence for using social-agency environments in instructional design. Multimedia 
programs can result in broader learning if the visual materials are combined with personalized spoken explanations, 
especially when the student is made a participant rather than an observer of the learning environment. 
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